Method, system and computer-readable media for rendering of three-dimensional model data based on characteristics of objects in a real-world environment

ABSTRACT

The disclosed technologies identify opportunities to display relevant three-dimensional (“3D”) model data within a real-world environment as a user wears a wearable device. The 3D model data can be associated with objects, or items, and the 3D model data rendered for display is relevant in the sense that the items are determined to be of interest to the user and the items fits within the real-world environment in which the user is currently located. For instance, the techniques described herein can recognize items typically found in a kitchen or a dining room of a user&#39;s house, an office space at the user&#39;s place of work, etc. The characteristics of the recognized items can be identified and subsequently analyzed together to determine preferred characteristics of a user. In this way, the disclosed technologies can retrieve and display an item that correlates to (e.g., matches) the preferred characteristics of the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalApplication No. 62/588,189, filed Nov. 17, 2017 and entitled “AugmentedReality, Mixed Reality, and Virtual Reality Experiences,” the entirecontents of which are incorporated herein by reference.

BACKGROUND

Conventionally, a user can view an object, or an item, of interest via aweb site on a display screen in a two-dimensional (“2D”) environment.For instance, the user may be researching information about the item, oreven further, the user may be interested in acquiring the item. In thesetypes of scenarios, the user experience is most likely limited to anonline experience.

A wearable device has the ability to display virtual content to a user,in an augmented reality (“AR”) environment. As use of wearable devicesbecomes more prevalent, it has become difficult to effectively displayvirtual content that not only is of interest to the user, but that also“fits” within an immersive real-world environment in which the user iscurrently located.

Consequently, the user can spend a considerable amount of time viewingthree-dimensional (“3D”) models of different items in order to find anitem that not only is of interest to the user, but that also fits withinthe immersive real-world environment in which the user is currentlylocated. This may unnecessarily utilize computing resources such asprocessing cycles, memory, and network bandwidth. Moreover, this mightresult in inadvertent or incorrect user input to the wearable devicerendering the 3D models of different items the immersive real-worldenvironment, which can also unnecessarily utilize computing resourcessuch as processing cycles, memory, and network bandwidth.

It is with respect to these and other technical challenges that thedisclosure made herein is presented.

SUMMARY

The techniques described herein identify opportunities to render anddisplay relevant three-dimensional (“3D”) model data within an immersivereal-world environment as a user wears a wearable device. The 3D modeldata can be associated with objects, or items, and the 3D model datarendered for display is relevant in the sense that the items aredetermined to be of interest to the user and the items fits within theimmersive real-world environment in which the user is currently located.In order to address the technical problems described briefly above, andpotentially others, the disclosed technologies can recognize physicalitems that exist in the immersive real-world environment in which theuser is currently located. For instance, the techniques described hereincan recognize common items typically found in a kitchen of a user'shouse, a dining room of the user's house, an office space at the user'splace of work, etc.

The characteristics of a set of items that exist in a real-worldenvironment can be identified and subsequently analyzed together todetermine preferred characteristics of a user. The user may be a personwearing a wearable device and/or a person to which the real-worldenvironment belongs (e.g., a child's bedroom, a mother's office, afather's work shop, etc.). In this way, the disclosed technologies canretrieve and render for display an item that correlates to (e.g.,matches) the preferred characteristics of the user.

In one example, an analysis of the characteristics may indicate a userpreference with regard to a price category such as an “expensive” pricecategory (e.g., the user spends amounts that are substantially morecompared to average prices when purchasing items), a “moderate” pricecategory (e.g., the user spends amounts that are within a price rangethat is close to average prices when purchasing items), an “inexpensive”price category (e.g., the user spends amounts that are substantiallyless compared to average prices when purchasing items), and so forth.

In another example, an analysis of the characteristics may indicate auser preference with regard to a size category such as a “large” sizecategory (e.g., the sizes of existing items are noticeably larger thanaverage sizes), a “medium” size category (e.g., the sizes of existingitems are in line or close to average sizes), a “small” size category(e.g., the sizes of existing items are noticeably smaller than averagesizes), and so forth.

In further examples, an analysis of the characteristics may indicate auser preference with regard to a particular brand of items and/or aparticular color of items. A “brand” can include a name of amanufacturer or producer, a name of a model, a design, a symbol, oranother recognizable or identifiable feature that distinguishes anorganization or a product from its rivals for customers thinking aboutpurchasing an item. In even further examples, the characteristics of theexisting items may be indicative of a preferred decorative theme orstyle for the real-world environment (e.g., a sports theme for a child'sbedroom, a ballerina theme for a child's bedroom, a cowboy theme for achild's bedroom, etc.).

Once the user preferences for the scanned real-world environment aredetermined, a virtual item with characteristics that correlate to (e.g.,match) the preferred item characteristics for the real-world environmentcan be retrieved and rendered for display in the real-world environment.Consequently, the virtual item is one that complements physical itemsthat already exist in the environment, and a frictionless approach toviewing and/or purchasing an item that is compatible with, and “fits”within, the real-world environment can be realized.

Aspects of the technologies disclosed herein can be implemented by awearable device, such as an AR device. For example, a user of such adevice might provide input indicating an interest to enter or activate amode enabling the rendering of 3D model data of recommended items.Accordingly, the wearable device may communicate with a system torecommend and display items to a user based on characteristics ofexisting items that are already physically present in a particularreal-world environment. The wearable device can be configured to scanthe real-world environment to recognize the existing items and toanalyze the existing items to determine user preferences for itemcharacteristics.

In various embodiments, the scan can be implemented in response to userinput that explicitly expresses a user interest in a specific item. Forexample, a user wearing a wearable device can enter the real-worldenvironment (e.g., a kitchen) and audibly request that the wearabledevice display a specific item (e.g., a toaster). Based on the userinput, the wearable device can scan the real-world environment, collectdata on the existing items in the real-world environment, identifycharacteristics of the existing items without any further input neededfrom the user, and/or determine preferred characteristics based on ananalysis of the identified characteristics.

In additional embodiments, the scan can be implemented in response touser input that amounts to a general request for one or more items. Ageneral request may not identify a specific item, but rather, maysuggest an event or reason for item recommendations to be displayedwithin a real-world environment. For example, a parent can enter achild's room and ask the wearable device to recommend a gift for thechild's upcoming birthday. In response to the general request, thewearable device can scan the real-world environment, collect data on theexisting items in the real-world environment, identify characteristicsof the existing items without any further input needed from the user,and/or determine preferred characteristics based on an analysis of theidentified characteristics.

Through implementations of the disclosed technologies, 3D models ofitems that are of interest to a user and that fit within a particularreal-world environment (e.g., a room in a house, an office in an officesuite, etc.) can be rendered within different AR environments. Thedisclosed technologies improve a user experience by identifying relevantopportunities to display content that is of particular interest to theusers. In this way, the disclosed technologies tangibly improvecomputing efficiencies with respect to a wide variety of computingresources that would otherwise be consumed and/or utilized by improvinghuman-computer interaction and by reducing the amount of processingcycles and storage required by previous solutions. Technical benefitsother than those specifically identified herein might also be realizedthrough implementations of the disclosed technologies.

It should be appreciated that the above-described subject matter can beimplemented as a computer-controlled apparatus, a computer-implementedmethod, a computing device, or as an article of manufacture such as acomputer-readable medium. These and various other features will beapparent from a reading of the following Detailed Description and areview of the associated drawings.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intendedthat this Summary be used to limit the scope of the claimed subjectmatter. Furthermore, the claimed subject matter is not limited toimplementations that solve any or all disadvantages noted in any part ofthis disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The Detailed Description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Thesame reference numbers in different figures indicate similar oridentical items.

FIG. 1 illustrates aspects of an exemplary computing environment inwhich a system can cause a 3D model of an item to be rendered, fordisplay, based on characteristics of items that already exist in areal-world environment in which a user is located.

FIG. 2 illustrates examples of preferred characteristics, which are usedto identify an item to be displayed via a wearable device of a user.

FIG. 3A illustrates an example where a user is looking at a portion of areal-world environment (e.g., a kitchen) while wearing a wearable deviceconfigured to recognize existing items and/or determine characteristicsof the existing items.

FIG. 3B illustrates an example of how a recommended item is rendered fordisplay based on user preferences with regard to item characteristics.

FIG. 4 illustrates aspects of another exemplary computing environment inwhich a system can cause a 3D model of an item to be rendered, fordisplay, based on characteristics of items that already exist in areal-world environment in which a user is located.

FIG. 5 illustrates aspects of yet another exemplary computingenvironment in which a system can cause a 3D model of an item to berendered, for display, based on characteristics of items that alreadyexist in a real-world environment in which a user is located.

FIG. 6 illustrates another example where a user is looking at a portionof a real-world environment (e.g., a kitchen) while wearing a wearabledevice configured to recognize existing items, determine characteristicsof the existing items, and recommend that an old item be replaced with anew item because the old item is nearing an end of its expected life.

FIG. 7 is a flow diagram that illustrates an example process describingaspects of the technologies disclosed herein for recognizing objectsthat exist in a real-world environment.

FIG. 8 is a flow diagram that illustrates an example process describingaspects of the technologies disclosed herein for determining preferredcharacteristics of a user based on the recognized objects.

FIG. 9 shows an illustrative configuration of a wearable device capableof implementing aspects of the technologies disclosed herein.

FIG. 10 illustrates additional details of an example computerarchitecture for a computer capable of implementing aspects of thetechnologies described herein.

DETAILED DESCRIPTION

This Detailed Description describes identifying opportunities to renderand display relevant three-dimensional (“3D”) model data within animmersive real-world environment as a user wears a wearable device. The3D model data can be associated with objects, or items, and the 3D modeldata rendered for display is relevant in the sense that the items aredetermined to be of interest to the user and the items fits within theimmersive real-world environment in which the user is currently located.

For instance, the techniques described herein can recognize itemstypically found in a kitchen of a user's house, a dining room of theuser's house, an office space at the user's place of work, etc. Thecharacteristics of the recognized items can be identified andsubsequently analyzed together to determine preferred characteristics ofa user. The user may be a person wearing a wearable device and/or aperson to which the real-world environment belongs (e.g., a child'sbedroom, a mother's office, a father's work shop, etc.). In this way,the disclosed technologies can retrieve and render for display an itemthat correlates to (e.g., matches) the preferred characteristics of theuser.

Referring now to the FIGS., technologies for efficiently rendering 3Dmodels of items for a user will be described.

FIG. 1 illustrates aspects of an exemplary computing environment 100 inwhich a system can cause a 3D model of an item to be rendered, fordisplay, based on characteristics of items that already exist in areal-world environment in which a user is located. As illustrated, theexemplary system may comprise an electronic commerce (“e-commerce”)system 102 that includes an item catalog 104 where users and/ormerchants can list real-world items for sale. A real-world item can beany type of item including, but not limited to, electronics, home goods,automobiles or automotive parts, clothing, musical instruments, art,jewelry, and so forth. In various examples, the e-commerce system 102can be implemented on one or more server computers operating inconjunction with of an e-commerce site.

A user 106 can utilize a wearable device 108, such as that described infurther detail below with respect to FIG. 9, to obtain image data 110 ofthe real-world environment in which the user 106 is currently located.For instance, the wearable device 108 can include an optical deviceconfigured to scan the real-world environment of the user 106 to obtainthe image data 110 (e.g., recognize objects in the real-worldenvironment). In various examples, the image data 110 of the real-worldenvironment includes recognizable existing items 112 that are physicallypresent in the real-world environment.

While the user 106 is in the real-world environment, the user mayprovide input (e.g., a voice command, a text entry in a search field, amenu option selection, etc.) that amounts to a request to view a virtualrepresentation of a specific item 114. For example, the user 106 canwalk into his or her kitchen and ask that the wearable device 108 todisplay a new toaster. Based on the request, the wearable device 108 canscan the real-world environment, recognize the existing items 112,identify characteristics 116 associated with the existing items 112, andsend the characteristics 116 to the e-commerce system 102 overnetwork(s) 118. For example, the characteristics 112 can include a priceof an item, a size of an item, a brand of an item, a color of an item, adecorative theme of a group of items, and so forth. A “brand” caninclude a name of a manufacturer or a producer, a name of a model, adesign, a symbol, or another recognizable or identifiable feature thatdistinguishes an organization or a product from its rivals for customersthinking about purchasing the item.

In some embodiments, the wearable device 108 can send the image data 110to the e-commerce system 102 and the e-commerce system 102 can recognizethe existing items 112 and identify the characteristics 116 associatedwith the existing items 112. For example, the e-commerce system 102 canlook up a price of a specific item via the item catalog.

Upon receiving the image data 110 and/or the characteristics 116, thee-commerce system 102 can employ a preferred characteristicsdetermination tool 120 (e.g., a software component or module) todetermine preferred characteristics 122 for the real-world environment.If the real-world environment is a particular room in a home, thepreferences can be those of the user 106, or another user that in thehome that typically occupies the room. For instance, the user 106 may bea mother or a father using the wearable device 108 to scan the bedroomof a child.

Based on an analysis of the characteristics 116 of the existing items112, the preferred characteristics determination tool 120 may determinea user preference for a particular brand (e.g., BOSCH, KITCHENAID, etc.)of kitchen items (e.g., appliances) in a kitchen. In another example,based on an analysis of the characteristics 116 of the existing items112, the preferred characteristics determination tool 120 may determinea user preference to spend amounts within a particular price categoryfor kitchen items in the kitchen. In yet another example, based on ananalysis of the characteristics 116 of the existing items 112, thepreferred characteristics determination tool 120 may determine a userpreference for a particular color or color scheme (e.g., white,stainless steel, etc.) of kitchen items in the kitchen. In even afurther example, based on an analysis of the characteristics 116 of theexisting items 112, the preferred characteristics determination tool 120may determine a user preference for a particular size of kitchen itemsin the kitchen.

In various embodiments, the preferred characteristics 122 may bedetermined relative to averages. Consequently, in some cases, apreferred characteristic 122 can comprise a category to which theassociated characteristics 116 of the existing items are mapped. Forexample, a preferred price characteristic 122 may include a pricecategory such as an “expensive” price category (e.g., the user spendsamounts that are substantially more compared to average prices whenpurchasing items), a “moderate” price category (e.g., the user spendsamounts that are within a price range that is close to average priceswhen purchasing items), an “inexpensive” price category (e.g., the userspends amounts that are substantially less compared to average priceswhen purchasing items), and so forth. In another example, a preferredsize characteristic 122 may include a size category such as a “large”size category (e.g., the sizes of existing items are noticeably largerthan average sizes), a “medium” size category (e.g., the sizes ofexisting items are in line or close to average sizes), a “small” sizecategory (e.g., the sizes of existing items are noticeably smaller thanaverage sizes), and so forth.

The e-commerce system 102 then uses an item correlation tool 124 toidentify an item in the item catalog 104 (e.g., an item available to bepurchased) that correlates to (e.g., matches) the preferredcharacteristics 122, the item being of the type specifically requestedby the user 106 (e.g., a toaster for the kitchen). That is, the itemcorrelation tool 124 may identify an item that is of the type requestedand that falls within a particular price category and/or a particularsize category. Moreover, the item may be associated with a preferredbrand, a preferred color, and/or a decorative theme of the kitchen.

Accordingly, the e-commerce system 102 is configured to retrieve 3Dmodel data 126 for the identified item and cause the 3D model data 126to be rendered for display via the wearable device 108 of the user 106.That is, the e-commerce system 102 transmits the 3D model data 126 tothe wearable device 108 so that the wearable device 108 can display arendering of the item 128, using the 3D model data, in a view of thereal-world environment 130 of the user 106. Consequently, in response tothe request to view a virtual representation of a specific item 114, theuser 106 is provided with a recommended item with characteristics thatcorrelate to (e.g., match) the characteristics 116 of the existing items112 that are already physically present in the real-world environment.This enables the user 106 to preview how the recommended item fitswithin the real-world environment before purchasing the item via theitem catalog 104.

In some embodiments, item metadata can be displayed with the renderingof the item 128. For instance, the item metadata can include, but is notlimited to: a name of the item, a description of the item (e.g., amanufacturer, a model, a size, etc.), a price for the item, and soforth.

An example of a wearable device 108 can include an augmented reality(“AR”) device. An AR device is a computing device capable of providing aview of the real-world environment 130 within which physical objects areaugmented or supplemented by computer-generated (“CG”) sensory input(e.g., sound, video, graphics, etc.). For instance, an AR device mightprovide a view of the real-world environment 130 with a rendering of anitem 128 as an overlay such that the item appears to be present in theview of real-world environment 130. Additional details regarding theconfiguration and operation of a wearable device 108 capable ofproviding this functionality is provided below with regard to FIG. 9. Inthis regard, it is to be appreciated that the rendering of an item 128by the wearable device 108 using the 3D model data includes displaying avirtual representation of the item in an AR environment, as well asother types of environments, such as mixed reality (“MR”) environmentsor virtual reality (“VR”) environments. It is also to be appreciatedthat the configurations disclosed herein are not limited to use with anAR device. Rather, the technologies disclosed herein can be utilizedwith any type of computing device that can provide a view of areal-world environment 130 that includes a rendering of an item 128.

It is to be further appreciated that the technologies described hereincan be implemented on a variety of different types of wearable devices108 configured with a variety of different operating systems, hardwarecomponents, and/or installed applications. In various configurations,for example, the wearable device 108 can be implemented by the followingexample wearable devices: GOOGLE GLASS, MAGIC LEAP ONE, MICROSOFTHOLOLENS, META 2, SONY SMART EYEGLASS, HTC VIVE, OCULUS GO, PLAYSTATIONVR, or WINDOWS mixed reality headsets. Thus, embodiments of the presentdisclosure can be implemented in any AR-capable device, which isdifferent than goggles or glasses that obstruct a user's view ofreal-world objects, e.g., actual reality. The techniques describedherein can be device and/or operating system agnostic.

FIG. 2 illustrates example preferred characteristics 200, which can beused to identify a virtual item to be displayed via a wearable device ofa user. As described above, the preferred characteristics determinationtool 120 can analyze the characteristics 116 of existing items 112 in areal-world environment to determine, as a preferred characteristics 200,a price category 202. For example, a preference can be mapped to one ofthe following price categories: an “expensive” price category 202A, a“moderate” price category 202B, or an “inexpensive” price category 202C.The e-commerce system 102 can establish the categories based on anaverage price of an item in the item catalog 104 and/or by referencingexternal sources (e.g., third-party retail or e-commerce sites).

Moreover, the e-commerce system 102 can establish the categories usingthresholds. For instance, the expensive price category 202A may berepresentative of a user typically spending more than a thresholdpercentage above the average prices (e.g., a user likes luxury items andtherefore the user typically spends more than 15% over the average priceto purchase an item), the moderate price category 202B may berepresentative of the user typically spending within a thresholdpercentage of the average prices (e.g., a user likes standard items andtherefore the user typically spends amounts within 15% of the averageprice to purchase an item), and the inexpensive price category 202C maybe representative of the user typically spending more than a thresholdpercentage below the average prices (e.g., a user likes discount itemsand therefore the user typically spends more than 15% less than theaverage price to purchase an item). In some instances, the thresholdpercentages can be based on a type of environment (e.g., dining rooms, achild's room, etc.) and/or a category of items.

Further, the preferred characteristics determination tool 120 cananalyze the characteristics 116 of existing items 112 in a real-worldenvironment to determine, as a preferred characteristics 200, a sizecategory 204. The e-commerce system 102 can establish the categoriesbased on an average size for a type of item (e.g., toaster, flat-screentelevision, etc.) in the item catalog 104 and/or by referencing externalsources. For example, a preference can be mapped to one of the followingsize categories: a “large” size category 204A representative of a userpreferring items that are a threshold percentage above the average sizes(e.g., the size of existing items are typically at least 10% larger thanaverage sizes), a “medium” size category 204B representative of the userpreferring items that are within the threshold percentage of the averagesizes (e.g., the size of existing items are typically within 10% of theaverage sizes), or a “small” size category 204C representative of theuser preferring items that are a threshold percentage below the averagesizes (e.g., the size of existing items are typically at least 10%smaller than average sizes).

Even further, the preferred characteristics determination tool 120 cananalyze the characteristics 116 of existing items 112 in a real-worldenvironment to determine, as a preferred characteristics 200, aparticular brand 206 (e.g., particular features such as logos, icons,shapes, etc. can be recognized and associated with a brand name and/ormodel), a particular color 208, and/or a particular decorative theme orstyle 210 (e.g., many of the existing items relate to sports, many ofthe existing items are rustic, many of the existing item relate to airplanes, etc.).

It is understood in the context if this disclosure that other itemcharacteristics can be used in order to determine user preferences andto find an item recommendation that matches the user preferences.Moreover, the names and number of categories described above areprovided as examples. In the context of this disclosure, it isunderstood that a characteristic can include more or less than threecategories and that an individual category can be named differently.Ultimately, categories may be created by the e-commerce system 102 sothat characteristics of existing items can be used to determine userpreferences and so that the user preferences can be used to identifyitems in which the user is likely interested. As described above, suchtechnologies save computing resources because the wearable device 108and/or the e-commerce system 102 are made aware of what user(s) prefer,and in turn, the wearable device 108 and/or the e-commerce system 102 nolonger have to display multiple item recommendations to find one thatthe user likes.

FIG. 3A illustrates an example 300 where a user 302 (e.g., user 106) islooking at a portion of a real-world environment (e.g., a kitchen) whilewearing a wearable device 304 (e.g., wearable device 108) configured torecognize existing items and/or determine characteristics of theexisting items. The view into the kitchen provided via the wearabledevice 304 comprises a real-world view from the perspective of the user302. As shown, the kitchen includes a coffee maker 306 and a microwave308, which are existing items physical present in the kitchen. In thisexample 300, the user 302 can provide input 310, such as an audiblecommand that states: “Show me a toaster for my kitchen”. In response,the wearable device 304 can be configured to scan the kitchen torecognize existing items which include the coffee maker 306 and themicrowave 308, as well as characteristics of the existing items (e.g., asize, a color, a brand, a price, etc.). The wearable device 304 may thentransmit the existing items and/or the characteristics to the e-commercesystem 102 so that the characteristics can be used to determine userpreferences.

FIG. 3B illustrates an example 312 of how a recommended item is renderedfor display based on the user preferences. As described above, via thenetwork(s) 118 and the e-commerce system 102, an item catalog 104 can beaccessed to identify an item the user is likely interested inpurchasing. The item can include characteristics that correlate to theuser preferences (e.g., same or similar price category, same or similarsize category, same or similar brand, same or similar color, an itemthat fits within a decorative theme, etc.). In the example 312, thewearable device 304 can virtually render a toaster 314 for display onthe kitchen counter between the coffee pot 306 and the microwave 308.The toaster 314 can have characteristics that are the same or similar tothose of the coffee pot 306 and the microwave 308. Moreover, itemmetadata for the toaster 314 can be displayed as well. In a specificexample, the item metadata can include different merchants and/or usersfrom which the toaster 314 can be purchased, as well as the prices(e.g., Company ABC is selling the toaster for $59, Vendor XYZ is sellingthe toaster for $55, Private User is selling the toaster for $50). Theitem metadata can be selectable so that the user can select one of thesellers of the toaster 314 and be taken to an item profile page of theseller so that the item can be directly purchased.

FIG. 4 illustrates aspects of another exemplary computing environment400 in which a system can cause a 3D model of an item to be rendered,for display, based on characteristics of items that already exist in areal-world environment in which a user is located. The environment 400of FIG. 4 is similar to the environment 100 in FIG. 1. However, whilethe environment 100 of FIG. 1 receives a request for a specific itemfrom the user 106 (e.g., the request clearly identifies an item ofinterest to the user), the environment 400 of FIG. 4 receives a generalrequest for one or more items 402 from the user 106.

A general request may not identify a specific item, but rather, maysuggest an event or reason for item recommendations to be displayedwithin a real-world environment. For example, a parent can enter achild's room and ask the wearable device 108 to recommend a gift for thechild's upcoming birthday. In response, the wearable device 108 can scanthe child's room to determine characteristics 116 of existing items 112and send the characteristics 116 to the e-commerce system 102 over thenetwork(s) 118. In another example, while viewing the living room in ahouse, a user can provide input that identifies a theme or a style alongwith a request for item recommendations. More specifically, the user maywant to entertain family and friends for an event, and thus, may requestthat the wearable device 108 virtually decorate the living room withitems for the event (e.g., Christmas, Thanksgiving, a birthday party, abig sports game between Team A and Team B, etc.).

As described above, upon receiving the characteristics 116 based on thegeneral request for one or more items 402, the e-commerce system 102 canemploy a preferred characteristics determination tool 120 to determinepreferred characteristics 122 for the real-world environment. That is,based on an analysis of the characteristics 116 of the existing items112, the preferred characteristics determination tool 120 may determinea preferred decorative theme or style for the real-world environment(e.g., a sports them for a child's room, a ballerina theme for a child'sroom, a cowboy theme for a child's room, etc.), a preferred brand ofitems (e.g., a child that loves sports may prefer UNDER ARMOUR overNIKE, etc.), a preferred price category for items (e.g., whetherresidents in the house typically purchase expensive or inexpensiveitems), a preferred color or color scheme for items, a preferred sizefor items, and so forth.

Based on the preferred characteristics 122, the e-commerce system 102can identify one or more suitable items that satisfy the general request402 received from the user 106 (e.g., an item that has characteristicsthat correlate to the preferred characteristics 122 and that are relatedto an event specified by the user). The e-commerce system 102 can thenretrieve 3D model data for a suitable item 404 from the item catalog andsend the 3D model data for the suitable item 404 to the wearable device108 so that it can be used to display a rendering of the suitable item406. Expanding on the examples provided above, the suitable item can bea child's birthday gift that complements existing items and/or fitswithin the decorative style or theme of the child's room. Moreover,multiple suitable items may virtually show the user 106 how the livingroom can be populated with items to reflect a theme or style associatedwith an event.

FIG. 5 illustrates aspects of yet another exemplary computingenvironment 500 in which a system can cause a 3D model of an item to berendered, for display, based on characteristics of items that alreadyexist in a real-world environment in which a user is located. Theenvironment 500 of FIG. 5 is similar to the environment 100 in FIG. 1.However, the environment 500 of FIG. 5 does not receive any requests foritems from the user 106. Rather the wearable device 108 and/or thee-commerce system 102 in FIG. 5 are configured to automatically andcontinually create and maintain an inventory of items 502 that belongto, or are owned by, the user 106 or residents in a home (e.g., afamily). The inventory of items 502 may include a current and/orpermanent location of an item in a space the user 106 resides or spendsa lot of time (e.g., an appliance in a kitchen, a flat screen televisionin a living room, a printer in an office, etc.).

For example, as the user 106 walks around his or her home while wearingthe wearable device 108, the wearable device 108 can perform objectrecognition to identify items owned by the user 106. The wearable device108 can then cause the identified items to be stored in an inventory 502(e.g., a home inventory, a work inventory, a vacation place inventory,etc.). In some instances, a separate inventory can be created andmaintained for individual real-world environments (e.g., an inventoryfor each of the living room, the garage, the kitchen, the masterbedroom, a secondary bedroom, etc.).

In the example of FIG. 5, the wearable device 108 and/or the e-commercesystem 102 is configured to proactively track and/or log informationuseable to determine if an item should be replaced. This information caninclude an age of an item (e.g., how old an item is), a total number ofuses of an item, a frequency of use of an item, and/or other informationindicative of usage. The wearable device 108 and/or the e-commercesystem 102 can use this information to determine an item that may needto be replaced 504 before it breaks or fails. Thus, the wearable device108 and/or the e-commerce system 102 can identify an opportune time torecommend that the user 106 replace the item (e.g., an old or anoverused item) with a new item.

In some examples, the wearable device 108 and/or the e-commerce system102 can access usage information (e.g., an expected life such as five orten years, an amount of use such as one hundred uses or one thousanduses, etc.) provided by a manufacturer of an item to determine when theitem may likely fail and/or break. Thus, the wearable device 108 and/orthe e-commerce system 102 can recommend that the item be replaced priorto a time when the item is likely to fail and/or break, or when the itemis nearing an end of its expected life.

The wearable device 108 and/or the e-commerce system 102 can then usethe preferred characteristics 122 to identify a new item to replace anold item and to retrieve 3D model data for the new item 506 from theitem catalog. Upon sending the 3D model data for the new item 506, thewearable device, in turn, can display a rendering of the new item 508 ina view of the real-world environment 130 using the 3D model data.

Using the example of FIGS. 3A and 3B as a starting point, FIG. 6illustrates another example 600 where a user 302 is looking at a portionof a real-world environment (e.g., a kitchen) while wearing a wearabledevice 304 configured to recognize existing items and/or determinecharacteristics of the existing items. In this example, the wearabledevice 304 is configured to determine that an existing toaster 402 isnearing the end of its life, and therefore, the wearable device 304and/or the e-commerce system 102 can recommend a new toaster 404. Thisdetermination can be made without any input from the user. Rather, thewearable device 304 can automatically track usage of the items and/orlog the usage in the inventory of items so that a new toaster can berecommended for purchase at a time when the old toaster should bereplaced. The new toaster 404 can have characteristics that are the sameor similar to those of the coffee pot 306 and the microwave 308.

To implement some of the described techniques on the wearable device108, a user may be required to enable a feature and/or enter aparticular operation mode. For example, the user 106 may need to providepermission and/or authorization for the wearable device 108 to implementthe described techniques.

FIGS. 7 and 8 are flow diagrams that each illustrate an example processdescribing aspects of the technologies presented herein with referenceto FIGS. 1-6. A process is illustrated as a collection of blocks in alogical flow graph, which represent a sequence of operations that can beimplemented in hardware, software, or a combination thereof. In thecontext of software, the blocks represent computer-executableinstructions that, when executed by one or more processors, perform therecited operations.

The particular implementation of the technologies disclosed herein is amatter of choice dependent on the performance and other requirements ofa computing device such as a wearable device. Accordingly, the logicaloperations described herein may be referred to variously as states,operations, structural devices, acts, or modules. These states,operations, structural devices, acts, and modules can be implemented inhardware, software (i.e. computer-executable instructions), firmware, inspecial-purpose digital logic, and any combination thereof. Generally,computer-executable instructions include routines, programs, objects,components, data structures, and the like that perform or implementparticular functions. It should be appreciated that more or feweroperations can be performed than shown in the figures and describedherein. These operations can also be performed in a different order thanthose described herein. Other processes described throughout thisdisclosure shall be interpreted accordingly.

FIG. 7 is a flow diagram that illustrates an example process 700describing aspects of the technologies disclosed herein for recognizingobjects that exist in a real-world environment and determining preferredcharacteristics of a user based on the recognized objects.

The process 700 begins at block 702, where user input is received. Theuser input can specify a request to display an item (e.g., a specificrequest, a general request, etc.). Based on the user input, the processproceeds to block 704 where an optical device of the wearable devicescans the real-world environment to obtain image data. At block 706, thewearable device may recognize items that already exist in the real-worldenvironment. The process 700 then proceeds to block 708 where the imagedata and/or the recognized items are sent to an e-commerce system. Atblock 710, 3D model data of an item that matches the requested item isreceived. At block 712, a rendering of the item is displayed on adisplay device of the wearable device using the 3D model data.

FIG. 8 is a flow diagram that illustrates an example process 800describing aspects of the technologies disclosed herein for determiningpreferred characteristics of a user based on the recognized objects.

The process 800 begins at block 802, where image data of a real-worldenvironment and/or recognized items that exist in the real-worldenvironment are received from a wearable device of a user. The processproceed to block 804 where the e-commerce system analyzes the real-worlditems that exist in the real-world environment to determine preferredcharacteristics for the real-world environment. At block 806, an itemcatalog is accessed to retrieve three-dimensional model data for an itemthat has characteristics that correlate to the preferredcharacteristics. At block 808, a rendering of the item is caused to bedisplayed on a display device of the wearable device of the user. Forinstance, 3D model data of the item is transmitted from the e-commercesystem 102 to the wearable device 108.

FIG. 9 shows an illustrative configuration of a wearable device 900(e.g., a headset system, a head-mounted display, etc.) capable ofimplementing aspects of the technologies disclosed herein. The wearabledevice 900 includes an optical system 902 with an illumination engine904 to generate electro-magnetic (“EM”) radiation that includes both afirst bandwidth for generating computer-generated (“CG”) images and asecond bandwidth for tracking physical objects. The first bandwidth mayinclude some or all of the visible-light portion of the EM spectrumwhereas the second bandwidth may include any portion of the EM spectrumthat is suitable to deploy a desired tracking protocol.

In the example configuration, the optical system 902 further includes anoptical assembly 906 that is positioned to receive the EM radiation fromthe illumination engine 904 and to direct the EM radiation (orindividual bandwidths of thereof) along one or more predeterminedoptical paths. For example, the illumination engine 904 may emit the EMradiation into the optical assembly 906 along a common optical path thatis shared by both the first bandwidth and the second bandwidth. Theoptical assembly 906 may also include one or more optical componentsthat are configured to separate the first bandwidth from the secondbandwidth (e.g., by causing the first and second bandwidths to propagatealong different image-generation and object-tracking optical paths,respectively).

The optical assembly 906 includes components that are configured todirect the EM radiation with respect to one or more components of theoptical assembly 906 and, more specifically, to direct the firstbandwidth for image-generation purposes and to direct the secondbandwidth for object-tracking purposes. In this example, the opticalsystem 902 further includes a sensor 908 to generate object data inresponse to a reflected-portion of the second bandwidth, i.e. a portionof the second bandwidth that is reflected off an object that existswithin a real-world environment.

In various configurations, the wearable device 900 may utilize theoptical system 902 to generate a composite view (e.g., from aperspective of a user 106 that is wearing the wearable device 900) thatincludes both one or more CG images and a view of at least a portion ofthe real-world environment that includes the object. For example, theoptical system 902 may utilize various technologies such as, forexample, AR technologies to generate composite views that include CGimages superimposed over a real-world view. As such, the optical system902 may be configured to generate CG images via a display panel. Thedisplay panel can include separate right eye and left eye transparentdisplay panels.

Alternatively, the display panel can include a single transparentdisplay panel that is viewable with both eyes and/or a singletransparent display panel that is viewable by a single eye only.Therefore, it can be appreciated that the technologies described hereinmay be deployed within a single-eye Near Eye Display (“NED”) system(e.g., GOOGLE GLASS) and/or a dual-eye NED system (e.g., OCULUS RIFT).The wearable device 900 is an example device that is used to providecontext and illustrate various features and aspects of the userinterface display technologies and systems disclosed herein. Otherdevices and systems may also use the interface display technologies andsystems disclosed herein.

The display panel may be a waveguide display that includes one or morediffractive optical elements (“DOEs”) for in-coupling incident lightinto the waveguide, expanding the incident light in one or moredirections for exit pupil expansion, and/or out-coupling the incidentlight out of the waveguide (e.g., toward a user's eye). In someexamples, the wearable device 1200 may further include an additionalsee-through optical component.

In the illustrated example of FIG. 9, a controller 910 is operativelycoupled to each of the illumination engine 904, the optical assembly 906(and/or scanning devices thereof) and the sensor 908. The controller 910includes one or more logic devices and one or more computer memorydevices storing instructions executable by the logic device(s) to deployfunctionalities described herein with relation to the optical system902. The controller 910 can comprise one or more processing units 912,one or more computer-readable media 914 for storing an operating system916 and data such as, for example, image data that defines one or moreCG images and/or tracking data that defines one or more object trackingprotocols.

The computer-readable media 914 may further include an image-generationengine 918 that generates output signals to modulate generation of thefirst bandwidth of EM radiation by the illumination engine 904 and alsoto control the scanner(s) to direct the first bandwidth within theoptical assembly 906. Ultimately, the scanner(s) direct the firstbandwidth through a display panel to generate CG images that areperceptible to a user, such as a user interface.

The computer-readable media 914 may further include an object-trackingengine 920 that generates output signals to modulate generation of thesecond bandwidth of EM radiation by the illumination engine 904 and alsothe scanner(s) to direct the second bandwidth along an object-trackingoptical path to irradiate an object. The object tracking engine 920communicates with the sensor 908 to receive the object data that isgenerated based on the reflected-portion of the second bandwidth.

The object tracking engine 920 then analyzes the object data todetermine one or more characteristics of the object such as, forexample, a depth of the object with respect to the optical system 902,an orientation of the object with respect to the optical system 902, avelocity and/or acceleration of the object with respect to the opticalsystem 902, or any other desired characteristic of the object. Thecomponents of the wearable device 900 are operatively connected, forexample, via a bus 922, which can include one or more of a system bus, adata bus, an address bus, a PCI bus, a Mini-PCI bus, and any variety oflocal, peripheral, and/or independent buses.

The wearable device 900 may further include various other components,for example cameras (e.g., camera 924), microphones (e.g., microphone926), accelerometers, gyroscopes, magnetometers, temperature sensors,touch sensors, biometric sensors, other image sensors, energy-storagecomponents (e.g. battery), a communication facility, a GPS receiver,etc. Furthermore, the wearable device 900 can include one or more eyegaze sensors 928. In at least one example, an eye gaze sensor 928 isuser facing and is configured to track the position of at least one eyeof a user. Accordingly, eye position data (e.g., determined via use ofeye gaze sensor 928), image data (e.g., determined via use of the camera924), and other data can be processed to identify a gaze path of theuser. That is, it can be determined that the user is looking at aparticular section of a hardware display surface, a particularreal-world object or part of a real-world object in the view of theuser, and/or a rendered object or part of a rendered object displayed ona hardware display surface.

In some configurations, the wearable device 900 can include an actuator929. The processing units 912 can cause the generation of a hapticsignal associated with a generated haptic effect to actuator 929, whichin turn outputs haptic effects such as vibrotactile haptic effects,electrostatic friction haptic effects, or deformation haptic effects.Actuator 929 includes an actuator drive circuit. The actuator 929 maybe, for example, an electric motor, an electro-magnetic actuator, avoice coil, a shape memory alloy, an electro-active polymer, a solenoid,an eccentric rotating mass motor (“ERM”), a linear resonant actuator(“LRA”), a piezoelectric actuator, a high bandwidth actuator, anelectroactive polymer (“EAP”) actuator, an electrostatic frictiondisplay, or an ultrasonic vibration generator.

In alternate configurations, wearable device 900 can include one or moreadditional actuators 929. The actuator 929 is an example of a hapticoutput device, where a haptic output device is a device configured tooutput haptic effects, such as vibrotactile haptic effects,electrostatic friction haptic effects, or deformation haptic effects, inresponse to a drive signal. In alternate configurations, the actuator929 can be replaced by some other type of haptic output device. Further,in other alternate configurations, wearable device 900 may not includeactuator 929, and a separate device from wearable device 900 includes anactuator, or other haptic output device, that generates the hapticeffects, and wearable device 900 sends generated haptic signals to thatdevice through a communication device.

The processing unit(s) 912, can represent, for example, a CPU-typeprocessing unit, a GPU-type processing unit, a field-programmable gatearray (“FPGA”), another class of digital signal processor (“DSP”), orother hardware logic components that may, in some instances, be drivenby a CPU. For example, and without limitation, illustrative types ofhardware logic components that can be used include Application-SpecificIntegrated Circuits (“ASICs”), Application-Specific Standard Products(“ASSPs”), System-on-a-Chip Systems (“SOCs”), Complex Programmable LogicDevices (“CPLDs”), etc.

As used herein, computer-readable media, such as computer-readable media914, can store instructions executable by the processing unit(s) 922.Computer-readable media can also store instructions executable byexternal processing units such as by an external CPU, an external GPU,and/or executable by an external accelerator, such as an FPGA typeaccelerator, a DSP type accelerator, or any other internal or externalaccelerator. In various examples, at least one CPU, GPU, and/oraccelerator is incorporated in a computing device, while in someexamples one or more of a CPU, GPU, and/or accelerator is external to acomputing device.

In various examples, the wearable device 900 is configured to interact,via network communications, with a network device (e.g., a networkserver or a cloud server) to implement the configurations describedherein. For instance, the wearable device 900 may collect data and sendthe data over network(s) to the network device. The network device maythen implement some of the functionality described herein. Subsequently,the network device can cause the wearable device 900 to display an itemand/or instruct the wearable device 900 to perform a task.

Computer-readable media can include computer storage media and/orcommunication media. Computer storage media can include one or more ofvolatile memory, nonvolatile memory, and/or other persistent and/orauxiliary computer storage media, removable and non-removable computerstorage media implemented in any method or technology for storage ofinformation such as computer-readable instructions, data structures,program modules, or other data. Thus, computer storage media includestangible and/or physical forms of media included in a device and/orhardware component that is part of a device or external to a device,including but not limited to random access memory (“RAM”), staticrandom-access memory (“SRAM”), dynamic random-access memory (“DRAM”),phase change memory (“PCM”), read-only memory (“ROM”), erasableprogrammable read-only memory (“EPROM”), electrically erasableprogrammable read-only memory (“EEPROM”), flash memory, rotating media,optical cards or other optical storage media, magnetic storage, magneticcards or other magnetic storage devices or media, solid-state memorydevices, storage arrays, network attached storage, storage areanetworks, hosted computer storage or any other storage memory, storagedevice, and/or storage medium that can be used to store and maintaininformation for access by a computing device.

In contrast to computer storage media, communication media can embodycomputer-readable instructions, data structures, program modules, orother data in a modulated data signal, such as a carrier wave, or othertransmission mechanism. As defined herein, computer storage media doesnot include communication media. That is, computer storage media doesnot include communications media consisting solely of a modulated datasignal, a carrier wave, or a propagated signal, per se.

In accordance with examples described herein, the wearable device 108can also be configured to use network communications to interact with ane-commerce provider of an electronic marketplace. To implement theelectronic marketplace, the e-commerce provider creates and maintainscatalog(s) of items. The items can be bought and/or sold by registeredusers and/or merchants. Accordingly, the e-commerce provider cancomprise resources to collect and store information related to an item,to display the information related to the item to a potential buyer, toconduct online auctions of an item, to match a buyer of an item with aseller of the item, to process a transaction, etc.

FIG. 10 shows additional details of an example computer architecture fora computer capable of executing the functionalities described hereinsuch as, for example, those described with reference to FIGS. 1-9, orany program components thereof as described herein. Thus, the computerarchitecture 1000 illustrated in FIG. 10 illustrates an architecture fora server computer, or network of server computers, or any other type ofcomputing device suitable for implementing the functionality describedherein. The computer architecture 1000 may be utilized to execute anyaspects of the software components presented herein, such as softwarecomponents for implementing the e-commerce system 102, including thepreferred characteristics determination tool 120 and/or the itemcorrelation tool 124.

The computer architecture 1000 illustrated in FIG. 10 includes a centralprocessing unit 1002 (“CPU”), a system memory 1004, including arandom-access memory 1006 (“RAM”) and a read-only memory (“ROM”) 1008,and a system bus 1010 that couples the memory 1004 to the CPU 1002. Abasic input/output system containing the basic routines that help totransfer information between elements within the computer architecture1000, such as during startup, is stored in the ROM 1008. The computerarchitecture 1000 further includes a mass storage device 1012 forstoring an operating system 1014, other data, and one or moreapplication programs. For example, the mass storage device 1012 maystore preferred characteristics 122, an inventory of items 502, as wellas 3D model data 1016 for items.

The mass storage device 1012 is connected to the CPU 1002 through a massstorage controller (not shown) connected to the bus 1010. The massstorage device 1012 and its associated computer-readable media providenon-volatile storage for the computer architecture 1000. Although thedescription of computer-readable media contained herein refers to a massstorage device, such as a solid-state drive, a hard disk or CD-ROMdrive, it should be appreciated by those skilled in the art thatcomputer-readable media can be any available computer storage media orcommunication media that can be accessed by the computer architecture1000.

According to various implementations, the computer architecture 1000 mayoperate in a networked environment using logical connections to remotecomputers through a network 1050. The computer architecture 1000 mayconnect to the network 1050 through a network interface unit 1018connected to the bus 1010. It should be appreciated that the networkinterface unit 1018 also may be utilized to connect to other types ofnetworks and remote computer systems. The computer architecture 1000also may include an input/output controller 1020 for receiving andprocessing input from a number of other devices, including a keyboard,mouse, or electronic stylus. Similarly, the input/output controller 1020may provide output to a display screen, a printer, or other type ofoutput device. It should also be appreciated that a computing system canbe implemented using the disclosed computer architecture 1000 tocommunicate with other computing systems.

It should be appreciated that the software components described hereinmay, when loaded into the CPU 1002 and executed, transform the CPU 1002and the overall computer architecture 1000 from a general-purposecomputing system into a special-purpose computing system customized tofacilitate the functionality presented herein. The CPU 1002 may beconstructed from any number of transistors or other discrete circuitelements, which may individually or collectively assume any number ofstates. More specifically, the CPU 1002 may operate as a finite-statemachine, in response to executable instructions contained within thesoftware modules disclosed herein. These computer-executableinstructions may transform the CPU 1002 by specifying how the CPU 1002transitions between states, thereby transforming the transistors orother discrete hardware elements constituting the CPU 1002.

Encoding the software modules presented herein also may transform thephysical structure of the computer-readable media presented herein. Thespecific transformation of physical structure may depend on variousfactors, in different implementations of this description. Examples ofsuch factors may include, but are not limited to, the technology used toimplement the computer-readable media, whether the computer-readablemedia is characterized as primary or secondary storage, and the like.For example, if the computer-readable media is implemented assemiconductor-based memory, the software disclosed herein may be encodedon the computer-readable media by transforming the physical state of thesemiconductor memory. For example, the software may transform the stateof transistors, capacitors, or other discrete circuit elementsconstituting the semiconductor memory. The software also may transformthe physical state of such components in order to store data thereupon.

As another example, the computer-readable media disclosed herein may beimplemented using magnetic or optical technology. In suchimplementations, the software presented herein may transform thephysical state of magnetic or optical media, when the software isencoded therein. These transformations may include altering the magneticcharacteristics of particular locations within given magnetic media.These transformations also may include altering the physical features orcharacteristics of particular locations within given optical media, tochange the optical characteristics of those locations. Othertransformations of physical media are possible without departing fromthe scope and spirit of the present description, with the foregoingexamples provided only to facilitate this discussion.

In light of the above, it should be appreciated that many types ofphysical transformations take place in the computer architecture 1000 inorder to store and execute the software components presented herein. Italso should be appreciated that the computer architecture 1000 mayinclude other types of computing devices, including smartphones,embedded computer systems, tablet computers, other types of wearablecomputing devices, and other types of computing devices known to thoseskilled in the art. It is also contemplated that the computerarchitecture 1000 may not include all of the components shown in FIG.10, may include other components that are not explicitly shown in FIG.10, or may utilize an architecture completely different than that shownin FIG. 10.

Illustrative Configurations

The following clauses described multiple possible configurations forimplementing the features described in this disclosure. The variousconfigurations described herein are not limiting nor is every featurefrom any given configuration required to be present in anotherconfiguration. Any two or more of the configurations may be combinedtogether unless the context clearly indicates otherwise. As used hereinin this document “or” means and/or. For example, “A or B” means Awithout B, B without A, or A and B. As used herein, “comprising” meansincluding listed all features and potentially including addition ofother features that are not listed. “Consisting essentially of” meansincluding the listed features and those additional features that do notmaterially affect the basic and novel characteristics of the listedfeatures. “Consisting of” means only the listed features to theexclusion of any feature not listed.

The disclosure presented herein also encompasses the subject matter setforth in the following clauses.

Example Clause A, a method comprising: recognizing, based on image dataobtained by a wearable device, real-world items that already exist inthe environment; analyzing the real-world items that already exist inthe environment to determine preferred characteristics for theenvironment, wherein the preferred characteristics include one or moreof a price category, a size category, a brand, a color, or a decorativetheme; accessing an item catalog to retrieve three-dimensional modeldata for an item that has characteristics that correlate to thepreferred characteristics; and causing a rendering of the item to bedisplayed on a display device of the wearable device using thethree-dimensional model data.

Example Clause B, the method of Example Clause A, wherein the preferredcharacteristics include the price category and the price category isdetermined relative to average prices of items in the item catalog.

Example Clause C, the method of Example Clause B, wherein the pricecategory comprises one of an expensive price category representative ofa user typically spending more than a threshold percentage above theaverage prices, a moderate price category representative of the usertypically spending within the threshold percentage of the averageprices, or an inexpensive price category representative of the usertypically spending more than a threshold percentage below the averageprices.

Example Clause D, the method of any one of Example Clauses A through C,wherein the preferred characteristics include the size category and thesize category is determined relative to average sizes of items in theitem catalog.

Example Clause E, the method of Example Clause D, wherein the sizecategory comprises one of a large size category representative of a userpreferring items that are a threshold percentage above the averagesizes, a medium size category representative of the user preferringitems that are within the threshold percentage of the average sizes, ora small size category representative of the user preferring items thatare a threshold percentage below the average sizes.

Example Clause F, the method of any one of Example Clauses A through E,wherein recognizing the real-world items that already exist in theenvironment and analyzing the real-world items to determine thepreferred characteristics for the environment are implemented inresponse to receiving a user input that specifically requests that theitem be displayed in the environment.

Example Clause G, the method of Example Clause F, wherein the user inputcomprises a voice command.

Example Clause H, the method of any one of Example Clauses A through E,wherein recognizing the real-world items that already exist in theenvironment and analyzing the real-world items to determine thepreferred characteristics for the environment are implemented inresponse to receiving a user input that generally requests for an itemrecommendation for a particular event.

Example Clause I, the method of Example Clause A, further comprisingaccessing information associated with a real-world item to determine thereal-world item is to be replaced, wherein the information comprises alife expectancy for the real-world item or an amount of item uses forthe real-world item, the rendering of the item to be displayed on thedisplay device of the wearable device corresponding to the real-worlditem to be replaced in the environment.

Example Clause J, the method of any one of Example Clauses A through I,wherein the environment comprises a type of room in a personal residenceor a business office.

Example Clause K, a system comprising: one or more processors; and amemory in communication with the one or more processors, the memoryhaving computer-readable instructions stored thereupon which, whenexecuted by the one or more processors, cause the one or more processorsto: recognize, based on image data obtained by a wearable device,real-world items that already exist in the environment; analyze thereal-world items that already exist in the environment to determinepreferred characteristics for the environment; access an item catalog toretrieve three-dimensional model data for an item that hascharacteristics that correlate to the preferred characteristics; andcause a rendering of the item to be displayed on a display device of thewearable device using the three-dimensional model data.

Example Clause L, the system of Example Clause K, wherein the preferredcharacteristics include a price category and the price category isdetermined relative to average prices of items in the item catalog.

Example Clause M, the system of Example Clause L, wherein the pricecategory comprises one of an expensive price category representative ofa user typically spending more than a threshold percentage above theaverage prices, a moderate price category representative of the usertypically spending within the threshold percentage of the averageprices, or an inexpensive price category representative of the usertypically spending more than a threshold percentage below the averageprices.

Example Clause N, the system of any one of Example Clauses K through M,wherein the preferred characteristics include a size category and thesize category is determined relative to average sizes of items in theitem catalog.

Example Clause O, the system of Example Clause N, wherein the sizecategory comprises one of a large size category representative of a userpreferring items that are a threshold percentage above the averagesizes, a medium size category representative of the user preferringitems that are within the threshold percentage of the average sizes, ora small size category representative of the user preferring items thatare a threshold percentage below the average sizes.

Example Clause P, the system of any one of Example Clauses K through O,wherein the preferred characteristics include a brand.

Example Clause Q, the system of any one of Example Clauses K through P,wherein the preferred characteristics include a decorative theme.

Example Clause R, the system of any one of Example Clauses K through Q,wherein recognizing the real-world items that already exist in theenvironment and analyzing the real-world items to determine thepreferred characteristics for the environment are implemented inresponse to receiving a user input that specifically requests that theitem be displayed in the environment.

Example Clause S, the system of any one of Example Clauses K through Q,wherein recognizing the real-world items that already exist in theenvironment and analyzing the real-world items to determine thepreferred characteristics for the environment are implemented inresponse to receiving a user input that generally requests for an itemrecommendation for a particular event.

Example Clause T, one or more non-transitory computer-readable mediahaving computer-readable instructions stored thereupon which, whenexecuted by one or more processors, cause a system to: recognize, basedon image data obtained by a wearable device, real-world items thatalready exist in the environment; analyze the real-world items thatalready exist in the environment to determine preferred characteristicsfor the environment, wherein the preferred characteristics include oneor more of a price category, a size category, a brand, or a decorativetheme; access an item catalog to retrieve three-dimensional model datafor an item that has characteristics that correlate to the preferredcharacteristics; and cause a rendering of the item to be displayed on adisplay device of the wearable device using the three-dimensional modeldata.

CONCLUSION

For ease of understanding, the processes discussed in this disclosureare delineated as separate operations represented as independent blocks.However, these separately delineated operations should not be construedas necessarily order dependent in their performance. The order in whichthe process is described is not intended to be construed as alimitation, and any number of the described process blocks may becombined in any order to implement the process or an alternate process.Moreover, it is also possible that one or more of the providedoperations is modified or omitted.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts are disclosed as example forms ofimplementing the claims.

The terms “a,” “an,” “the” and similar referents used in the context ofdescribing the invention (especially in the context of the followingclaims) are to be construed to cover both the singular and the pluralunless otherwise indicated herein or clearly contradicted by context.The terms “based on,” “based upon,” and similar referents are to beconstrued as meaning “based at least in part” which includes being“based in part” and “based in whole” unless otherwise indicated orclearly contradicted by context.

It should be appreciated that any reference to “first,” “second,” etc.users or other elements within the Summary and/or Detailed Descriptionis not intended to and should not be construed to necessarily correspondto any reference of “first,” “second,” etc. elements of the claims.Rather, any use of “first” and “second” within the Summary and/orDetailed Description may be used to distinguish between two differentinstances of the same element (e.g., two different users, two differentitems, etc.).

Certain configurations are described herein, including the best modeknown to the inventors for carrying out the invention. Of course,variations on these described configurations will become apparent tothose of ordinary skill in the art upon reading the foregoingdescription. Skilled artisans will know how to employ such variations asappropriate, and the configurations disclosed herein may be practicedotherwise than specifically described. Accordingly, all modificationsand equivalents of the subject matter recited in the claims appendedhereto are included within the scope of this disclosure. Moreover, anycombination of the above-described elements in all possible variationsthereof is encompassed by the invention unless otherwise indicatedherein or otherwise clearly contradicted by context.

What is claimed is:
 1. A method comprising: recognizing, based on imagedata obtained by a wearable device, real-world items that already existin an environment; determining preferred characteristics for theenvironment based on the real-world items recognized in the image dataobtained by the wearable device, wherein the preferred characteristicsdetermined based on the real-world items recognized in the image dataobtained by the wearable device include a user preference for aparticular price category; accessing an item catalog to retrievethree-dimensional model data for an item that has characteristics thatcorrelate to the preferred characteristics and is within the particularprice category; and causing a rendering of the item to be displayed on adisplay device of the wearable device using the three-dimensional modeldata.
 2. The method of claim 1, wherein the particular price category isdetermined relative to average prices of items in the item catalog. 3.The method of claim 2, wherein the particular price category comprisesone of an expensive price category representative of a user typicallyspending more than a threshold percentage above the average prices, amoderate price category representative of the user typically spendingwithin the threshold percentage of the average prices, or an inexpensiveprice category representative of the user typically spending more than athreshold percentage below the average prices.
 4. The method of claim 1,wherein the preferred characteristics further include a size categorythat is determined relative to average sizes of items in the itemcatalog.
 5. The method of claim 4, wherein the size category comprisesone of a large size category representative of a user preferring itemsthat are a threshold percentage above the average sizes, a medium sizecategory representative of the user preferring items that are within thethreshold percentage of the average sizes, or a small size categoryrepresentative of the user preferring items that are a thresholdpercentage below the average sizes.
 6. The method of claim 1, whereinrecognizing the real-world items that already exist in the environmentand the determining the preferred characteristics for the environmentare implemented in response to receiving a user input that specificallyrequests that the item be displayed in the environment.
 7. The method ofclaim 6, wherein the user input comprises a voice command.
 8. The methodof claim 1, wherein recognizing the real-world items that already existin the environment and the determining the preferred characteristics forthe environment are implemented in response to receiving a user inputthat generally requests for an item recommendation for a particularevent.
 9. The method of claim 1, further comprising accessinginformation associated with a real-world item to determine thereal-world item is to be replaced, wherein the information comprises alife expectancy for the real-world item or an amount of item uses forthe real-world item, the rendering of the item to be displayed on thedisplay device of the wearable device corresponding to the real-worlditem to be replaced in the environment.
 10. The method of claim 1,wherein the environment comprises a type of room in a personal residenceor a business office.
 11. A system comprising: one or more processors;and a memory in communication with the one or more processors, thememory having computer-readable instructions stored thereupon which,when executed by the one or more processors, cause the one or moreprocessors to: recognize, based on image data obtained by a wearabledevice, real-world items that already exist in an environment; determinepreferred characteristics for the environment based on the real-worlditems recognized in the image data obtained by the wearable device,wherein the preferred characteristics determined based on the real-worlditems recognized in the image data obtained by the wearable deviceinclude a user preference for a particular price category; access anitem catalog to retrieve three-dimensional model data for an item thathas characteristics that correlate to the preferred characteristics andis within the particular price category; and cause a rendering of theitem to be displayed on a display device of the wearable device usingthe three-dimensional model data.
 12. The system of claim 11, whereinthe particular price category is determined relative to average pricesof items in the item catalog.
 13. The system of claim 12, wherein theparticular price category comprises one of an expensive price categoryrepresentative of a user typically spending more than a thresholdpercentage above the average prices, a moderate price categoryrepresentative of the user typically spending within the thresholdpercentage of the average prices, or an inexpensive price categoryrepresentative of the user typically spending more than a thresholdpercentage below the average prices.
 14. The system of claim 11, whereinthe preferred characteristics further include a size category and thesize category is determined relative to average sizes of items in theitem catalog.
 15. The system of claim 14, wherein the size categorycomprises one of a large size category representative of a userpreferring items that are a threshold percentage above the averagesizes, a medium size category representative of the user preferringitems that are within the threshold percentage of the average sizes, ora small size category representative of the user preferring items thatare a threshold percentage below the average sizes.
 16. The system ofclaim 11, wherein the preferred characteristics further include a brand.17. The system of claim 11, wherein the preferred characteristicsfurther include a decorative theme.
 18. The system of claim 11, whereinrecognizing the real-world items that already exist in the environmentand determining the preferred characteristics for the environment areimplemented in response to receiving a user input that specificallyrequests that the item be displayed in the environment.
 19. The systemof claim 11, wherein recognizing the real-world items that already existin the environment and determining the preferred characteristics for theenvironment are implemented in response to receiving a user input thatgenerally requests for an item recommendation for a particular event.20. One or more non-transitory computer-readable media havingcomputer-readable instructions stored thereupon which, when executed byone or more processors, cause a system to: recognize, based on imagedata obtained by a wearable device, real-world items that already existin the environment; determine preferred characteristics for anenvironment based on the real-world items recognized in the image dataobtained by the wearable device, wherein the preferred characteristicsdetermined based on the real-world items recognized in the image dataobtained by the wearable device include a user preference for aparticular price category; access an item catalog to retrievethree-dimensional model data for an item that has characteristics thatcorrelate to the preferred characteristics and is within the particularprice category; and cause a rendering of the item to be displayed on adisplay device of the wearable device using the three-dimensional modeldata.